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1.  Monthly  Mean  Error  Patterns 

Figures  1,2,  and  3  depict  the  monthly  mean  error  patterns  for 
24-hour  PE  sea-level  pressure  progs  for  January,  February,  and 
March  1971,  respectively.  Figures  4,  5,  and  6  show  the  correspond¬ 
ing  48-hour  error  patterns. 

Figures  7,8,  and  9  depict  the  monthly  mean  error  patterns  for 
24-hour  PE  500  MB  height  progs  for  January,  February,  and  March 
1971,  respectively.  Figures  10,  11,  and  12  show  the  corresponding 
48-hour  error  patterns. 

Recalling  that  monthly  mean  error  patterns  are  designed  to 
show  systematic  bias  in  the  model,  note  the  improving  trend  during 
the  period  in  the  24-hour  surface  progs  as  shown  in  Figures  1,  2,  and 
3.  In  general,  most  of  the  systematic  errors  were  located  over 
land  areas  in  February  and  March.  Figures  4,5,  and  6  also  show 
improving  trends  in  the  48-hour  progs,  particularly  so  in  the  Atlantic 
and  Mediterranean. 

Figures  7,8,  and  9  indicate  this  same  improving  trend  at  500  MBS. 
The  persistent  negative  bias  over  Northwest  Africa  and  over  the  south¬ 
western  U.S.  has  been  very  difficult  to  diagnose.  Elsewhere,  the 
magnitudes  are  smaller  and  considerably  easier  to  evaluate. 


Figures  10,11  and  12  show  that  we  still  have  significant 
systematic  errors  by  48-hours,  at  500  MBS.  Over  Africa,  this 
amounts  to  about  -90  meters.  Elsewhere,  the  biases  are  generally 
less  than  60  meters. 

2.  Verification  Statistics 

Table  1  contains  the  RMS  scores  (in  millibars)  for  PE  Model 
sea-level  pressure  progs.  It  is  important  to  note  that  the  model 
is  more  courageous  than  the  SLP  in  that  it  tends  to  predict  about 
the  same  amount  of  change  (in  an  RMS  sense)  that  actually  occurs. 
Comparing  the  actual  change  (persistence  error)  to  the  forecast 
error,  one  sees  that  the  model  contains  a  lot  of  skill  to  36  hours 
and  some  skill  even  after  72  hours.  In  the  forecast  error  column, 
note  the  remarkable  consistency  from  month-to-month  for  each  fore¬ 
cast  tau. 

Table  2  shows  the  model's  performance  in  terms  of  percent 
improvement  over  persistence,  summarized  by  tau  and  month.  Table 
3  has  the  same  statistics  for  the  SLP  Model  (until  it  was  no  longer 
run  in  parallel  with  the  PE).  The  differences  speak  for  themselves. 

Table  4  illustrates  the  degree  to  which  the  model  is  predicting 
the  observed  changes  (courage  index).  The  five-month  averages 
(by  tau)  show  that  we  are  very  close  in  this  regard.  For  any  par¬ 
ticular  month  and  tau,  the  model  generally  predicts  the  amount  of 
change  within  about  6%. 
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Project  FAMOS  compared  the  FNWC  PE  Model  to  the  NMC  PE 
Model  during  the  month  of  January.  Verification  statistics  (skill 
score  and  average  error)  were  kept  for  36-hour  sea-level  pressure 
progs  at  60  points  in  both  the  Atlantic  (25N-55N,  65W-15E)  and 
Pacific  (25N-55N,  135E-145W).  In  general,  the  FNWC  Model 
was  more  skillful  than  the  NMC  Model  in  the  Pacific,  but  slightly 
worse  in  the  Atlantic.  Recalling  that  a  smaller  skill  score  is  better, 
the  results  were: 


FNWC 

NMC 

Atlantic 

57.9 

57. 

Pacific 

57.3 

59. 

Any  score  less  than  60  was  considered  a  good  forecast. 

A  recent  paper  by  David  P.  Baumhefner  entitled  "Global  Real- 
Data  Forecasts  with  the  NCAR  Two-Layer  General  Circulation  Model" 
(NWR,  Volume  98,  Number  2)  helps  us  keep  our  perspective.  This 
rather  complex  model  was  used  for  a  series  of  four-day  forecasts 
using  real  data.  The  following  quote  is  extracted: 

"By  the  time  the  forecast  has  reached  48  hours,  the  skill 
shown  by  the  surface  pressure  prediction  was  minimal.  The  central 
pressure  of  the  three  major  storms  in  the  Northern  Hemisphere  is  in 
error  by  over  30  millibars. " 


3 


3.  Error  Patterns  as  a  Diagnostic  Tool 

Figures  13,  14,  and  15  represent  the  mean  sea-level  pressure 
distributions  in  January,  February,  and  March,  respectively.  Figures 
16,  17,  and  18  are  the  corresponding  mean  500  MB  distributions. 

Analyst- forecasters  using  PE  Model  guidance  materials  (progs) 
should  visually  correlate  the  mean  height  and  pressure  patterns 
with  the  appropriate  mean  error  patterns  as  an  exercise.  Caution 
is  advised  in  putting  too  much  faith  in  error  patterns  in  poor  data  areas, 
especially  the  500  MB  patterns  over  oceans.  A  few  illustrations 
follow. 

a .  Mediterranean  Area 

Note  the  500  MB  means  for  the  Mediterranean  region  as  shown 
in  Figures  16,  17,  and  18.  In  both  January  and  February  the  trough 
was  located  in  the  central  Med,  but  upstream  the  ridging  was  much 
more  pronounced  in  February.  Thus,  flows  tended  to  cross  the 
mountains  at  a  larger  angle  in  February.  Next,  look  at  the  48-hour 
surface  pressure  errors  in  Figures  10,  11,  and  12.  The  systematic 
negative  bias  shown  in  February  is  significantly  worse  (as  a  consequence) 
In  March,  however,  the  trough  shifted  position  westward  to  a  line 
from  Spain  down  the  Moroccan  coast,  with  quasi-zonal  flow  across 
the  Med.  The  systematic  error  essentially  disappeared. 

COMMENT.  We  realize  that  both  model  changes  and  seasonal 
factors  are  pertinent  to  a  discussion  of  this  nature.  Additionally,  we 
feel  that  the  problem  is  intimately  connected  with  the  inability  to 
model  the  complex  topographic  variations  and  land-water  relationships 
that  exist  in  this  region  with  a  coarse  grid. 
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b.  Eastern  United  States  Area 

Using  the  same  chart  set,  note  the  500  MB  patterns  over  the 
continental  U.S.  Flow  differences  are  at  a  minimum.  In  March  the 
eastern  Pacific  ridging  weakened  somewhat,  and  in  February  the  mean 
trough  was  located  more  toward  the  central  U0S.  California  cut-off 
lows  (troughs)  were  less  prevalent  in  March. 

From  Figures  13,  14,  and  15  one  sees  rather  large  differences  in 
the  surface  pressure  distributions.  In  January,  for  example,  the  mean 
ridge  extending  from  the  Dakotas  southeastward  to  the  Gulf  was 
stronger  in  the  mean  than  in  subsequent  months. 

Yst ,  Figures  4,  5,  and  6  indicate  that  model  bias  was  minimal 
over  the  U.S.  One  is  tempted  to  conclude  that  the  Rocky  Mountains, 
being  of  much  larger  horizontal  extent  than  the  European  mountains, 
are  much  less  difficult  to  treat  in  atmospheric  models. 

Critical  evaluation  of  U.S.  region  forecasts  during  the  quarter 
do  indicate  the  following  shortcomings  in  the  model: 

(1)  under-translation  of  pressure  systems  (see  PE  Progress 
Report  2-71) . 

(2)  slight  over-development  of  surface  highs  over  continents. 

(3)  slight  under-development  of  lows  which  spawn  in  the  lee 

of  the  southern  Rockies.  (This  is  more  noticeable  in  "dry"  small-scale 
storms .) 

(4)  over-development  of  Gulf  of  Mexico  lows  in  the  which  the 
middle  tropospheric  westerlies  overlay  moist  tropical  air.  (The  main 
problem  is  related  to  an  overactive  convective  precipitation  mechanism) 
but  other  factors  play  a  role.  These  are  poor  initialization,  proximity 

to  the  restoration  boundary,  and  the  general  inadequacy  of  dynamic 
methods  in  these  latitudes.) 
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c.  Western  United  States  and  Eastern  Pacific 
The  basic  problem  in  this  region  is  lack  of  upper  air  data  to 
initialize  the  model  in  the  eastern  Pacific.  But  other  factors  may¬ 
be  of  importance. 

Note  the  large,  persistent  negative  errors  at  500  MBS  in  Figures 
10,  11,  and  12.  These  patterns  are  as  difficult  to  evaluate  as  those 
shown  over  northwest  Africa,  but  may  in  fact  be  related.  By  this 
we  mean  that  both  of  these  negative  biases  coincide  with  kinetic 
energy  sources  for  the  three-wave  (hemispheric)  subtropical  jet 
(STJ) .  We  suspect  that  the  missing  error  pattern  (which  would  be 
over  China)  may  well  be  non-existent  because  we  have  good  upper 
air  coverage.  (This  STJ  hypothesis  has  not  been  tested) . 

Other  factors  which  are  relevant  are  proximity  to  restoration 
boundaries,  and  possible  geostrophic  adjustment  problems  for  flows 
which  are  quasi-parallel  to  topography  contours. 

These  are  just  a  few  illistrations  of  possible  uses  of  these  charts. 

The  reader  is  invited  to  study  the  error  patterns  carefully,  and  supplement 
them  with  his  own  notes  on  regional  model  problems. 

4.  Model  Changes  During  the  Quarter 
a  .  6  January  1971 

(1)  eliminated  bullseyes  in  some  upper  tropospheric  height 
progs  caused  by  scaling  spills. 

(2)  eliminated  bad  spots  on  the  surface  prog  borders. 
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(3)  put  in  routine  to  ellipticize  surface  pressure  progs 
small-scale  highs, 
b  .  26  January  1971 

(1)  shifted  PE  internal  boundary  from  17N  to  UN  in 
an  attempt  to  reduce  the  unrealistic  growth  of  sub¬ 
tropical  surface  pressure  gradients  (experimental 
change) . 

(2)  used  a  Laplacian-type  smoother  between  equator  and 
23N  on  output  surface  pressure  progs. 

c.  8  February  1971 

(1)  took  out  ellipticizor  and  replaced  it  with  a  simple 
computational  routine  to  fractionally  reduce  the 
pressure  profiles  of  highs  greater  than  1020  MBS. 

(2)  eliminated  topography  south  of  9  degrees  North. 

(3)  devised  and  coded  a  different  type  of  restoration 
boundary  condition,  the  effect  of  which  was  to  restore 
(after  an  integration  time  step,  and  in  the  tropical 
restoration  area  only)  the  values  of  the  variables 
back  toward  their  values  at  the  previous  time  step 
(rather  than  toward  their  initial  value) .  The  gleostrophic 
adjustment  process  was  enhanced  by  this,  and  did 
lead  to  a  much  better  control  of  sub-tropical  and  mid¬ 
latitude  highs  and  the  associated  peripheral  gradients 
on  the  south  side. 
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COMMENT.  All  of  the  ramifications  of  this  coding  change  were 
not  understood  until  mid-March.  Since  the  so-called  "heating 
package"  temperature  changes  are  calculated  once  an  hour  (after 
the  temperatures  are  restored  in  the  boundary  region) ,  restoration 
back  to  the  previous  value  vice  initial  value  inadvertently  caused 
systematic  cooling  in  the  upper  atmosphere  south  of  23N.  But 
because  of  this  coding  oversight  we  were  able  to  detect  that  the 
long-wave  radiational  losses  were  excessive.  It  is  our  feeling 
that  the  February  results  were  below  par  for  this  reason, 
d.  13  March  1971 

(1)  devised  and  coded  a  diagnostic-type  tropopause  in 
the  output  temperature/height  structure,  based  on 
a  regression  equation  by  Eady. 

COMMENT.  Tropopauses  generally  occur  between  the  model's  two 
uppermost  computational  levels  on  which  temperatures  are  known. 
Distributions  of  temperatures  (and  heights  between  300  MBS  and 
100  MBS  were  constructed  assuming  temperatures  varied  linearly  in 
log-pressure  between  computational  levels.  This,  of  course,  led 
to  incorrect  density  and  geostrophic  wind  forecasts  at  250- ,  200- 
and  150-  MBS . 

The  tropopause  pressure  is  computed  as  a  function  of  the  300- 
500  MB  and  150-100MB  thicknesses  at  each  output  tau.  It  is  con¬ 
strained  to  lie  between  400  MBS  and  120  MBS,  but  varies  in  space- 
time  consistent  with  the  meteorological  situation. 
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5.  Model  Changes  in  the  Near  Future 

a.  Re-partitioning  of  PE  Model 

At  present,  we  partition  the  computational  burden  to  achieve 
parallelism.  By  this,  we  mean  that  the  various  equations  are 
evaluated  in  different  processors,  but  each  processor  does  its  job 
on  the  entire  horizontal- vertical  domain.  In  the  near  future  we  will 
code  a  four- process  or  model  which  will  do  everything  the  current 
four- process  or  model  does,  but  each  processor  will  calculate  the 
results  for  one-fourth  of  the  grid  points.  By  this  we  hope  to  achieve 
a  speed-up  of  10-15%  (this  amounts  to  a  savings  of  two  processor- 
hours  per  day) . 

b.  Divergent  Initial  Winds 

The  final  touches  are  being  put  on  the  solution  of  a  three- 
dimensional  omega  equation  from  which  the  Chi— winds  will  be  computed. 
Earlier  experiments  indicated  that  the  Chi-winds  are  not  too  important 
for  short  progs,  but  become  increasingly  important  as  the  forecast 
period  lengthens .  Further ,  by  starting  the  model  with  initial  vertical 
velocities,  the  time  required  for  adjustment  of  the  mass-motion  fields 
is  minimized . 

c .  Terra  in 

As  discussed  earlier,  optimum  specification  of  the  underlying 
terrain  is  a  difficult,  complex  task.  One  fact  is  clear;  it  is  point¬ 
less  (and  even  detrimental)  to  represent  scale  features  in  the  terrain 
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that  the  grid  and  finite-differencing  scheme  cannot  resolve.  Further, 
we  know  that  by  over- smoothing  we  can  produce  unrealistic  meteo¬ 
rological  patterns.  By  including  too  much  terrain  realism  (precipitous 
terrain  gradients;  island  spikes;  knife-edge  ranges)  computational 
instabilities  result. 

In  any  event,  more  developmental  effort  in  this  respect  is 
necessary.  We  hope  to  have  this  problem  minimized  before  the  end 
of  the  summer  season. 

d.  Precipitation  Rates 

During  the  past  several  months  precipitation  patterns  have, 
on  occasion,  been  unrealistic.  Sometimes  the  trouble  appeared  as 
a  small  bullseye;  on  the  other  occasions  abnormally  heavy  bands  of 
precipitation  occurred  near  the  subtropical  internal  boundary. 

In  the  scheme  which  parameterizes  cumulus  clouds  and  precip¬ 
itation,  several  empirical  controls  can  be  adjusted.  For  example, 
we  can  regulate  the  degree  of  conditional  instability  before  precip¬ 
itation  and  adjustment  takes  place.  At  present,  this  criterion  is 
uniform  over  the  entire  hemisphere.  Thus,  we  can  tune  the  mecha  - 
nism  for  optimum  realism. 
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e.  Wind  Verification 

Programs  are  being  tested  at  present  to  verify  both  the 
independent  PE  winds  and  the  derived  geostrophic  winds  from  the 
model.  These  winds  will  be  compared  to  the  observed  winds  over 
North  America  at  the  850- ,  500-  and  300-  millibar  levels. 

The  results  will  be  summarized  for  bi-weekly  or  monthly  periods, 
and  be  expressed  in  terms  of  vector  mean  wind  errors  at  each  veri¬ 
fying  station  together  with  some  measure  of  error  vector  dispersal 
(standard  deviation),  by  wind  type,  forecast  length,  and  level. 

f.  Staggered  Grid 

The  problem  of  truncation  error  was  addressed  in  PE  Progress 
Report  2-71.  Recall  that  two  alternatives  were  discussed:  shifting 
to  fourth-order  differencing;  and/or  computing  on  a  finer  mesh;  noting 
that  the  latter  was  not  possible  with  our  present  computers. 

Actually,  a  third  possibility  might  well  be  considered;  that  is, 
reprogram  the  model  using  a  staggered  grid.  In  one  such  type  of 
scheme,  the  wind  components  are  carried  at  odd-numbered  points 
(say)  and  mass-field  parameters  at  even-numbered  points.  Other 
more  complicated  schemes  have  been  devised  and  summarized  in 
the  literature. 

The  essential  points  are  that  both  the  truncation  error  and 
the  central  memory  requirements  can  be  significantly  reduced. 
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Table  1 


RMS  VERIFICATION  OF  PE  MODEL  SURFACE  PRESSURE  PROGS 

(by  month  and  tau;  in  millibars) 


Forecast 

Tau  Month  Cha  nqe 


Actual 

Change 


12  NOV 

DEC 
JAN 
FEB 
MAR 

24  NOV 

DEC 
JAN 
FEB 
MAR 

36  NOV 

DEC 
JAN 
FEB 
MAR 

48  NOV 

DEC 
JAN 
FEB 
MAR 

60  NOV 

DEC 
JAN 
FEB 
MAR 

72  NOV 

DEC 
JAN 
FEB 
MAR 


4.4 

4.3 

4.6 

4.3 

4.7 

4.7 

4.4 

4.3 

4.3 

4.3 

6.8 

6.9 

7.0 

6.7 

7.4 

7.3 

6.7 

6.8 

6.5 

6.8 

8.3 

8.3 

00 

cn 

8.2 

9.1 

9.1 

8.3 

8.4 

o 

00 

LO 

CO 

9.2 

9.3 

9.4 

9.0 

10.3 

10.4 

9.4 

9.2 

8.9 

9.3 

10.1 

10.1 

10.1 

9.7 

11.2 

10.9 

10.0 

9.8 

9.5 

00 

CD 

10.4 

10.2 

10.5 

10.2 

11.4 

11.4 

10.8 

10.3 

00 

CD 

10.2 

Forecast 

Error 

3.0 

3.0 

3.0 

3.0 

2.9 

4.5 

4.6 
4.4 

4.4 
4.2 

5.9 
5.9 
5.9 
5.9 

5.5 

7.1 

7.1 

7.1 

7.2 
6.8 

8.5 

7.8 
8.0 

8.4 

7.9 

8.9 

8.6 
8.8 

9.5 
8.9 
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Table  2 


PE  MODEL  PERFORMANCE 

PERCENT  IMPROVEMENT  OVER  PERSISTENCE 

(SEA  LEVEL  PRESSURE  PROGS) 


Tau 

NOV  1970 

DEC  1970 

IAN  1971 

FEB  1971 

MAR  19 

12 

30.2 

30.2 

36.2 

30.4 

33.6 

24 

34.8 

31.4 

39.5 

34.7 

37.7 

36 

29.0 

28.1 

35.2 

29.9 

35.2 

48 

24.8 

21.2 

31.8 

21.9 

27.7 

60 

15.0 

19.6 

26.6 

14.1 

19.2 

72 

11.7 

15.7 

22.8 

7.7 

13.6 

13 


Table  3 


SLP  MODEL  PERFORMANCE 
PERCENT  IMPROVEMENT  OVER  PERSISTENCE 

Tau  November  1970  December  1970  January  1971 


12 

18.6 

23.2 

19.1 

24 

16.0 

16.4 

16.4 

36 

8.4 

7.3 

8.8 

48 

1.1 

1.1 

3.8 

60 

-5.0 

-2.1 

-4.6 

72 

-10.9 

-11.8 

-9.7 

NOTES:  (1)  The  SLP  Model  has  not  been  run  in  parallel  with  the  PE 
Model  since  January. 

(2)  A  negative  sign  indicates  that  persistence  was  better. 
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Table  4 


PE  MODEL  PERFORMANCE 

FORECAST  CHANGE  AS  A  PERCENTAGE 

OF  ACTUAL  CHANGE 


Tau 

NOV  1970 

DEC  1970 

TAN  1971 

FEB  1971 

MAR  1971 

AVERAGE 

12 

102 .3 

106.9 

100.0 

102 .8 

100.0 

102  .4 

24 

98.6 

105.2 

101 .4 

99.3 

94.3 

100.2 

36 

100.0 

103.6 

100.0 

99.0 

93.4 

99.2 

48 

99.0 

104.4 

99.9 

101.7 

95.6 

100.1 

60 

100.0 

103.0 

102.7 

102.0 

97.1 

101.0 

72 

101.9 

102.9 

100.0 

105.3 

96.0 

101.2 

NOTE:  A  number  101.5  indicates  that  the  Model  predicted  1.5%  more 
change  in  an  RMS  sense  that  actually  occurred. 
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Fsgure  3, 


January  1971  M ean  Error 
Patterns  for  24 “Hour  PE  Surface 
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Figure  2.  February  1971  Mean  Error 
Patterns  for  24-Hour  PE  Surface 
Pressure  Progs 
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Figure  6 «  March  1971  Mean  Errcif 

Pattern  a  for  A  3- Hour  PE  Surface 

Pressure  Progs 
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Figure  8.  February  1971  Mean  Error 
Patterns  for  24-Hour  PE  500  MB 
Progs 


Figure  9.  March  197]  Mean  Error 
Patterns  for  2  4 -Hour  PE  £00  MB 
Progs 
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Figure  10.  January  1971  Mean  Error 
Patterns  for  48-Hour  PE  500  MB 
Progs 
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March  1971  Mean  Error 
48-Hour  PE  500  MB 
Progs 


fLcrure  13,  January  1971 
Surf acp  Pressure  Analysis 


‘igurE  14,  February  197]  Mwn 
Surface  Pressure  Analysis 
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Figure  16.  January  1971  Mean 
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Figure  17.  February  1971  Mean 


500  MB  Analysis 
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